Not all features are created equal

TL;DR Recent studies by Anthropic show that LLM features extracted via mechanistic interpretability fall into distinct categories, each with different properties. However, state-of-the-art auto-interpreters fail to account for this variety. In this article, I propose AIR (Auto-Interpretability Route…

View Post on Scour ↗

More from lesswrong.com on Scour.

Covers The Biology of a Large Language Model